Module documentation based testing using grey-box approach

Testing plays an important role to assure the quality of software. Testing is a process of detecting errors that can be highly effective if performed rigorously. The use of formal specifications provides significant opportunity to develop effective testing techniques. Grey-box testing approach usually based on knowledge obtains from specification and source code while seldom the design specification is concerned. In this paper, we propose an approach for testing a module with internal memory from its formal specification based on grey-box approach. We use formal specifications that are documented using Parnas’s Module Documentation (MD) method. The MD provides us with the information of external and internal view of a module that can be useful in grey-box testing approach

Utilizing an abstraction relation document in grey-box testing approach

This paper describes the test oracle generation from an abstraction relation document that is documented using Parnas’s Module Documentation (MD) method. This work is part of on-going research that addresses the problem of improving the effectiveness of fault detection. We focus our work on unit/module testing where each module may consist of several programs. The aim of our project is to investigate the strategies and techniques to automate module testing. In particular, we investigate the use of MD that is written in standard mathematical notation to automate the process of test oracle generation and test execution

The automatic assessment of Z specifications

The need to automate the process of assessing a specification in a learning environment is identified to be one of the fundamental ways to improve the use of formal notation in specifying a real system. General issues involved in building an automatic marking system for computer-based courses are explored. Techniques that have been proposed for assessing a specification are also discussed. By considering the issues and the techniques, we describe how they can be used to build a system that is able to give a quality grade to a specification that is written in the Z language. In the system, four quality factors are taken into consideration; maintainability of a specification (which considers the typographic arrangement of a specification and the specification complexity), and correctness of a specification (which reflects the static correctness and the dynamic correctness of a specification). By using suitable quality metrics for specification maintainability, the results that are produced are compared to some values which can either be absolute values or relative to the model answer. The marks awarded for this factor are based on this comparison. Static correctness is carried out by applying a syntax and type checker. The marks granted for this factor depend on the outcome of the checker. Dynamic correctness is determined by employing a testing technique. In the context of a specification, the behaviour of a system-state, which is represented by so-called state variables, is analysed. The specification is 'executed' by using animation. The marks are given according to the correctness of the output and the final state. The system is implemented within the well-known courseware management system, Ceilidh. There are fundamental differences between Z specifications, and the subject matter of other courses taught using the Ceilidh system (which are mostly computer programming courses). For this reason we take some time in this thesis to explain (in some detail) the incorporation of the system within Ceilidh. The need for the fundamental components (i.e the editor, the syntax and type checker, the animator and the automatic marker) are discussed and described. The system has been used by a group of 13 students who attended a Z course within the School of Computer Science and Information Technology at the University of Nottingham during the 1997-1998 academic year. The students were given a questionnaire about the system. An analysis of these questionnaires shows that the currently implemented tools are beneficial and helpful to the students. We also test the results of the system and compare them with a small selected group of human markers. The testing reveals very encouraging results and shows that the system can mark student scripts with a good degree of accuracy. We conclude that this system can provide a very useful aid for teachers of the Z Specification language

The automatic assessment of Z specifications

The need to automate the process of assessing a specification in a learning environment is identified to be one of the fundamental ways to improve the use of formal notation in specifying a real system. General issues involved in building an automatic marking system for computer-based courses are explored. Techniques that have been proposed for assessing a specification are also discussed. By considering the issues and the techniques, we describe how they can be used to build a system that is able to give a quality grade to a specification that is written in the Z language. In the system, four quality factors are taken into consideration; maintainability of a specification (which considers the typographic arrangement of a specification and the specification complexity), and correctness of a specification (which reflects the static correctness and the dynamic correctness of a specification). By using suitable quality metrics for specification maintainability, the results that are produced are compared to some values which can either be absolute values or relative to the model answer. The marks awarded for this factor are based on this comparison. Static correctness is carried out by applying a syntax and type checker. The marks granted for this factor depend on the outcome of the checker. Dynamic correctness is determined by employing a testing technique. In the context of a specification, the behaviour of a system-state, which is represented by so-called state variables, is analysed. The specification is 'executed' by using animation. The marks are given according to the correctness of the output and the final state. The system is implemented within the well-known courseware management system, Ceilidh. There are fundamental differences between Z specifications, and the subject matter of other courses taught using the Ceilidh system (which are mostly computer programming courses). For this reason we take some time in this thesis to explain (in some detail) the incorporation of the system within Ceilidh. The need for the fundamental components (i.e the editor, the syntax and type checker, the animator and the automatic marker) are discussed and described. The system has been used by a group of 13 students who attended a Z course within the School of Computer Science and Information Technology at the University of Nottingham during the 1997-1998 academic year. The students were given a questionnaire about the system. An analysis of these questionnaires shows that the currently implemented tools are beneficial and helpful to the students. We also test the results of the system and compare them with a small selected group of human markers. The testing reveals very encouraging results and shows that the system can mark student scripts with a good degree of accuracy. We conclude that this system can provide a very useful aid for teachers of the Z Specification language

Comprehensive Aspectual UML Approach to Support AspectJ

Unified Modeling Language is the most popular and widely used Object-Oriented modelling language in the IT industry. This study focuses on investigating the ability to expand UML to some extent to model crosscutting concerns (Aspects) to support AspectJ. Through a comprehensive literature review, we identify and extensively examine all the available Aspect-Oriented UML modelling approaches and find that the existing Aspect-Oriented Design Modelling approaches using UML cannot be considered to provide a framework for a comprehensive Aspectual UML modelling approach and also that there is a lack of adequate Aspect-Oriented tool support. This study also proposes a set of Aspectual UML semantic rules and attempts to generate AspectJ pseudocode from UML diagrams. The proposed Aspectual UML modelling approach is formally evaluated using a focus group to test six hypotheses regarding performance; a “good design” criteria-based evaluation to assess the quality of the design; and an AspectJ-based evaluation as a reference measurement-based evaluation. The results of the focus group evaluation confirm all the hypotheses put forward regarding the proposed approach. The proposed approach provides a comprehensive set of Aspectual UML structural and behavioral diagrams, which are designed and implemented based on a comprehensive and detailed set of AspectJ programming constructs

An experimental assessment of module documentation-based testing

Context: Testing a module that has memory using the black-box approach has been found to be expensive and relatively ineffective. Instead, testing without knowledge of the specifications (white-box approach) may not be effective in showing whether a program has been properly implemented as stated in its specifications. We propose instead a grey-box approach called Module Documentation-based Testing or MD-Test, the heart of which is an automatic generation of the test oracle from the external and internal views of the module. Objective: This paper presents an empirical analysis and comparison of MD-Test against three existing testing tools. Method: The experiment was conducted using a mutation-testing approach, in two phases that assess the capability of MD-Test in general and its capability of evaluating test results in particular. Results: The results of the general assessment indicate that MD-Test is more effective than the other three tools under comparison, where it is able to detect all faults. The second phase of the experiment, which is significant to this study, compares the capabilities of MD-Test and JUnit-black using the test evaluation results. Likewise, an analysis of the test evaluation results shows that MD-Test is more effective and efficient, where MD-Test is able to detect at least the same number of faults as, or is at par with, the black-box approach. Conclusion: It is concluded that test evaluation using grey-box approach is more effective and efficient that the black-box approach when testing a module that has memory

Evaluation and extracting factual software architecture of distributed system by process mining techniques

The factual software architectures that are actually implemented of distributed systems do not conform the planned software architectures (Beck 2010). It happens due to the complexity of distributed systems. This problem begets two main challenges; First, how to extract the factual software architectures with the proper techniques and second, how to compare the planned software architecture with the extracted factual architecture. This study aims to use process mining to discover factual software architecture from codes and represents software architecture model in Petri Net to evaluate model by the linear temporal logic and process mining. In this paper, the applicability of process mining techniques, implemented in the ProM6.7 framework is shown to extract and evaluate factual software architectures. Furthermore, capabilities of Hierarchical Colored Petri Net implemented in CPN4.0 are exploited to model and simulate software architectures. The proposed approach has been conducted on a case study to indicate applicability of the approach in the distributed data base system. The final result of the case study indicates process mining is able to extract factual software architectures and also to check its conformance